Statistical tests for biological interactions: A comparison of permutation tests and analysis of variance

Interaction terms from statistical tests are often used to make inferences about biological processes. For interaction terms to be biologically meaningful, it is critical that the statistical method used tests a model that corresponds to a realistic null hypothesis. A commonly used data analysis method, the analysis of variance F-test (ANOVA), is limited when examining interactions because there are a limited number of statistical models that it can test. This is further complicated by the fact that data transformations, which affect the model being tested, are sometimes required to meet ANOVA’s assumptions. Thus, when using ANOVA, it can be difficult to determine whether interactions that are found in data were produced by biological mechanisms or are statistical artifacts due to an unrealistic model. A survey of the literature indicated that these shortcomings are often not recognized despite ANOVA’s widespread use. In this paper, we evaluate the suitability of an alternate method, permutation tests, compared to ANOVA. We compare the range of potential statistical models that each method can test and the power of each method to detect interactions when using an appropriate model. We provide two simulated experiments on species interactions that show that ANOVA and permutation tests have similar power when testing an appropriate statistical model, but that permutation tests provide an advantage over ANOVA in their ability to test a wider range of models. We conclude that permutation tests can be used to make inferences, potentially impossible with ANOVA, concerning biological interactions. a 2007 Elsevier Masson SAS. All rights reserved.